A researcher built a fictional billing database and had five AI models create semantic layers (business logic definitions) for it, then tested how well those layers helped the models answer finance questions. The results show that a well-built layer can dramatically improve accuracy, but models often make confident mistakes about scope and conventions that human finance teams would catch.

• Haiku failed to build a usable layer at all; the four models that succeeded got the math right but disagreed on scope—which accounts count as customers, whether subsidiaries roll up to parents, and how to handle QA test data.
• A reference layer written by a human improved Opus from 17 of 30 correct answers to 30 of 30, while a flawed layer built by Sonnet dropped Fable from 18 correct to 12, showing that semantic layers can either unlock or degrade model performance.
• Models built layers with high confidence regardless of accuracy: Muse matched finance definitions on all but one rule, while Opus and Fable made the same incorrect assumption about subsidiary rollups that finance never used, and Sonnet inconsistently filtered QA accounts across different metrics.

Organizations face three recurring data problems—inconsistent metric definitions, fragmented access controls, and metric changes that don't propagate everywhere. A semantic layer solves this by centralizing metric definitions and governance in one place, so all tools pull the same numbers and changes cascade automatically. It won't fix bad data at the source, but it shrinks the surface area you need to manage and makes self-service analytics actually work.

• The same metric getting defined differently across Tableau, Power BI, and Python isn't a minor annoyance—it's a root cause of bad executive decisions.
• Hiring more BI analysts as gatekeepers just creates ticket queues and bottlenecks; it doesn't fix fragmented governance across tools.
• A semantic layer centralizes metric definitions and access controls so a single change (e.g., redefining ARR) propagates automatically everywhere instead of requiring manual updates across systems.
• Centralizing definitions and logic in one place also makes the data self-documenting, enabling real self-service instead of ticket-based requests.

The LangChain data team replaced their old BI tool with Hex to power an agent-first data stack. By embedding models, metric definitions, business context, and trust signals, the data agent now handles about 40× more queries while the team focuses on high-impact projects.

• LangChain replaced its BI tool with Hex, combining notebooks, dashboards, and a conversational agent accessible via UI, Slack, CLI, and MCP
• Rich dbt model documentation plus a semantic layer defining key metrics let the agent generate trustworthy, company-aligned SQL and analysis
• The agent now handles ~40x the query volume the 3-person data team could manage, with ~2,200 conversations/month (23 per user) across a third of the company
• Freed from routine queries, the data team now focuses on refining models, adding guardrails, and tackling higher-leverage analytical work

Rill’s Metrics SQL lets you define business metrics once and query them using plain SQL across dashboards, notebooks, and AI agents. It compiles metric views into optimized OLAP queries, handling grouping, filters, time functions, and security automatically.

• Define a metric once in a YAML-backed metrics view and query it as plain SQL from dashboards, notebooks, and AI agents alike, instead of redefining it in dbt, Looker, Python, etc.
• A parser/compiler/executor pipeline rewrites simple queries like "SELECT country, revenue FROM revenue_metrics" into full OLAP SQL with correct GROUP BY, joins, and filters automatically.
• Security (row-level and column permissions) and SQL-injection-safe parameter binding are enforced consistently at the executor stage for every consumer.
• Works across multiple backends (ClickHouse, Snowflake, DuckDB, Druid) and can leverage existing database optimizations like materialized views and indexes for performance.